Statistical Active Learning Algorithms for Noise Tolerance and Differential Privacy

We describe a framework for designing efficient active learning algorithms that are tolerant to random classification noise and are differentially-private. The framework is based on active learning algorithms that are statistical in the sense that they rely on estimates of expectations of functions of filtered random examples. It builds on the powerful statistical query framework of Kearns (1993). We show that any efficient active statistical learning algorithm can be automatically converted to an efficient active learning algorithm which is tolerant to random classification noise as well as other forms of "uncorrelated" noise. The complexity of the resulting algorithms has information-theoretically optimal quadratic dependence on $1/(1-2\eta)$, where $\eta$ is the noise rate. We show that commonly studied concept classes including thresholds, rectangles, and linear separators can be efficiently actively learned in our framework. These results combined with our generic conversion lead to the first computationally-efficient algorithms for actively learning some of these concept classes in the presence of random classification noise that provide exponential improvement in the dependence on the error $\epsilon$ over their passive counterparts. In addition, we show that our algorithms can be automatically converted to efficient active differentially-private algorithms. This leads to the first differentially-private active learning algorithms with exponential label savings over the passive case.Comment: Extended abstract appears in NIPS 201

Robust Interactive Learning

In this paper we propose and study a generalization of the standard active-learning model where a more general type of query, class conditional query, is allowed. Such queries have been quite useful in applications, but have been lacking theoretical understanding. In this work, we characterize the power of such queries under two well-known noise models. We give nearly tight upper and lower bounds on the number of queries needed to learn both for the general agnostic setting and for the bounded noise model. We further show that our methods can be made adaptive to the (unknown) noise rate, with only negligible loss in query complexity

Combinatorial Auctions Do Need Modest Interaction

We study the necessity of interaction for obtaining efficient allocations in subadditive combinatorial auctions. This problem was originally introduced by Dobzinski, Nisan, and Oren (STOC'14) as the following simple market scenario: $m$ items are to be allocated among $n$ bidders in a distributed setting where bidders valuations are private and hence communication is needed to obtain an efficient allocation. The communication happens in rounds: in each round, each bidder, simultaneously with others, broadcasts a message to all parties involved and the central planner computes an allocation solely based on the communicated messages. Dobzinski et.al. showed that no non-interactive ($1$-round) protocol with polynomial communication (in the number of items and bidders) can achieve approximation ratio better than $\Omega(m^{{1}/{4}})$, while for any $r \geq 1$, there exists $r$-round protocols that achieve $\widetilde{O}(r \cdot m^{{1}/{r+1}})$ approximation with polynomial communication; in particular, $O(\log{m})$ rounds of interaction suffice to obtain an (almost) efficient allocation. A natural question at this point is to identify the "right" level of interaction (i.e., number of rounds) necessary to obtain an efficient allocation. In this paper, we resolve this question by providing an almost tight round-approximation tradeoff for this problem: we show that for any $r \geq 1$, any $r$-round protocol that uses polynomial communication can only approximate the social welfare up to a factor of $\Omega(\frac{1}{r} \cdot m^{{1}/{2r+1}})$. This in particular implies that $\Omega(\frac{\log{m}}{\log\log{m}})$ rounds of interaction are necessary for obtaining any efficient allocation in these markets. Our work builds on the recent multi-party round-elimination technique of Alon, Nisan, Raz, and Weinstein (FOCS'15) and settles an open question posed by Dobzinski et.al. and Alon et. al

Item Pricing for Revenue Maximization in Combinatorial Auctions

Consider the problem of a retailer with various goods for sale, attempting to set prices to maximize revenue. If customers have separate valuations over the different goods, and these are known to the retailer, then the goods can be priced separately and the problem is not so difficult. However, when customers have valuations over sets of items, this becomes a combinatorial auction problem, and the problem becomes computationally hard even when valuations are fully known in advance. In this talk we present some simple randomized algorithms and mechanisms for a number of interesting cases of this problem, both in the limited and unlimited supply setting. This talk is based on joint work with Avrim Blum and Yishay Mansour